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HUMINT Evaluation in the Intelligence Commumity 


E.0, 12036 requires that the heads of NFIP departments and agencies 
ensure that the DCI be provided, in a timely and responsive manner, with 
all information necessary to perform his program and budget responsibilities, 
With respect to evaluation, such information includes that which OMB 
designates (in OMB Circular No, A-1l, 25 May 1978) as: (a) changes 
in quality of output, (b) demonstrated results of past program 
activities, and (c) the results of benefit/cost and other pertinent | 
evaluation and agsigeia 

At present, two major Intelligence Community collectors of HUMINT, 

CIA and DoD, are reasonably well equipped to provide the required information 
on reporting quality. One, CIA, can provide certifiable aviaenes of actual 
results from a substantial amount of its reporting. Neither CIA nor any 
other NFIP or non-NFIP collector, however, has a system that can truly 
evaluate the apridieney. let alone the effectiveness, of a collection 
program by weighing quantified HUMINT output (results/effects of 

reporting) against HUMINT input (costs). This situation, which reflects 

the inherent difficulties involved in such quantification, is now being 
reviewed at the interagency level, with a view to sharing experience, 
acquiring new methodology, and better coordinating system development, 

A recent Intelligence Community seminar on HUMINT evaluation 
detailed past systems shortfalls, progress toward reporting evaluation 
that contributes indirectly to the pudeee process, and desirable steps 
toward compliance with the directives cited above on direct evaluation af 


input to the budget process, Among the conclusions reached at the seminar 


Approved For Release 2005/12/14 : CIA-RDP83M00171R001500160001-4 
SECRET 


SECRET 
2 Approved For @eace 2005/12/14 : CIA-RDP83M0017 @01500160001-4 


was the statement that CTS (HUMINT Tasking Office) leadership is essential 
in gaining the Intelligence Community support needed for truly effective 
evaluation, The following description of the CIA/DDO HUMINT evaluation 
system, and those of State and DIA in comparison with it, brings ‘out 
similarities and remaining contrasts among them. 
Development of CIA/DDO Evaluation System 

The present CIA/DDO reporting evaluation system was developed in 
response to changing needs. Collectors in the field, stretched thin 
by shrinking resources and broadening demands, had come to need increasing 
amounts of evaluative feedback in order to optimize their reporting 
efforts, Equally hard pressed, however, analyst consumers and policy/program 
users of the reporting had come increasingly to begrudge the time and 
effort spent in mechanistic, "box-checking," feedback programs. By 1975, 
these DDO programs had thus fallen largely into disuse, while at the 
Same time the budget-related needs of central management for HUMINT 
evaluation had begun to escalate. There was a clear need for improvement 
and extension of the DDO's personalized, substantive contact with customers, 
a ee ee ee ee ee oe 
This need has been met over the past three years with the result that the 
individual collector in the field, who initiates the evaluation process, 
is now part of a complete loop that runs through the operating management 
level in the DDO and includes on a regular basis the key consumers of CIA's 
HUMINT within the intelligence, foreign affairs, and U.S, military R&D 


program communities, 


25X1 
Quality Measurement at the Operating Level oa 
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valuation fo ntr Aa. me rpose 
The quality measurement (for operating, line-management purposes) 
described above constitutes only the base of the pyramid of information 


support required by the program-and-budget process. In the HUMINT flow 


system that produces foreign intelligence reports or acquires foreign 
Materiel for intelligence purposes, the determination of efficiency 
requires that evaluation tnelide comparine such output (acceptable, useful 
reports) to input. For aeaeRad management purposes, such comparisons need 
to include work measurement and/or unit cost on the input side, In order 
to move towards true effectiveness measurement, the output side must 

be filled in either with actual dollar values or a meaningful equivalent 
("shadow-price dollars"). Because of the subjective nature of a large 
portion of HUMINT collection - such as politioal reporting - it probably 
never will be possible to apply the same precise measurements that are 


25X1 


frequently applicable to technical collection, 


with concentration on the streams of reporting that have led some- 
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where, or failed to do so. A single, basic set of criteria for 
evaluation is used in the questioning, which begins with the ; 
identification of utility in the reporting, or lack thereof, It °; 
is pursued for the purpose of pinning down reporting results in 

terms of actual effect upon the production of finished intelligence, — 
the formulation of policy or its implementation, and the eonduct of 
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expensive programs in the U.S. military R&D community. 25X1 
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Better Measurement of Efficiency and Effectiveness 

The essence of the zero-base approach to budgeting is identification 
and evaluation of alternative methods of Sdeoapitahing objectives. According 
to OMB (Circular No, A-115, 5 May 1978), assessments of alternatives 
by decision unit managers in agencies and departments should be based 
on the relative efficiency of each alternative in accomplishing major 
objectives, Evaluation efforts should provide a basis for meaningful 
comparisons and the identification of alternatives should at least reflect 
a conscientious and continuing effort by management to improve the 
effectiveness of the units, Clearly, HUMINT evaluation must move as far 
above simple quality measurement as it can toward providing evidence of output 
values that can be matched, where it is meaningful to do so, against input 
cost. At a minimum, this requires more standardization of evaluation criteria 
than has so far taken place among HUMINT evaluators, as well as the development 
of more compatibility among their data bases than now exists. Beyond been 


cost-effectiveness evaluation and analysis must draw upon more careful 
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The attached memo was prepared by 


James Bush of - —_ = = —_ 7 Represen= 


tatives Permanent Select Committee on Intelligence. 
It is one of a series of short studies or memos 
which the Office of HUMINT Tasking has prepared 
for the "general education" of Bush. 

You may wish to route it on to D/DCI/RM 
for general background. 
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